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PEER REVIEW AND OPENNESS OF INFORMATION IN THE PROCESS OF 
MAKING ENVIRONMENTAL AND OTHER TECHNICAL DECISIONS 


Introduction 


The question of how social decisions are to be nen is as old as human 
society itself; the question of how decisions are to be made wisely is as 
old as philosophy itself; but the question of how technical decisions may 
best be made is comparatively new. 

Throughout most of history, the social goal has been decisions considered 
morally righteous according to one canon or another; or which were held to 
be correct simply because pronounced by an infallible monarch. Then, as 
the divine rights of kings and clerics yielded to the age of reason, the 
goal in decision-making tended to shift from righteousness and infallibility 
to justice, fairness, and equity. 

With the Industrial Revolution and age of technology, there arose a 
need for a iow dimension in societal decision-making. The need was not 
altogether without precedent. When humanoids evolved into homo faber -~ man 
the toolmaker -- it was necessary that spears and arrows were designed in 
such a way they flew tees oR and that cutting and crushing instruments 
performed their intended functions. In the Middle Ages, it was necessary to 
design cathedrals in such a way they stood rather than collapsed of their 
own weight. But with the Industrial Revolution, there exploded in propor- 
tion to the explosion of technological capability a need for decisions which 
were true, not = a metaphysical sense, but in an objective sense; decisions 
based not on utopian thinking or dogma or nares logic, but on accurate fore- 
knowledge of what the consequences would be. Should a factory be built 


upstream, close to the raw materials, or was the channel too shallow for 
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the larger ships required? Could a river be spanned, given the materials 
and bridgebuilding skill available? Where should roads be built in view 
of population trends? What type of roads in view of developments in trans- 


portation technology? 


There is almost always a “social lag" between the rates at which tech- 


nological and political institutions change. Industrial societies were 
generally slow to recognize that the new types of decisions being made had 
social implications so profound they could not safely be left to private 
entrepeneurs or private institutions any more than decisions in war could 
safely be left to generals. For decades, even generations, captains of 
industry were essentially free to build whatever kinds of factories, mines, 
railroads, or other facilities they wished, wherever they wished, and to 
operate them as heedlessly as they wished. Landholders could do whatever 
they wished with the land, lumbermen with the forests, hunters with the 
birds and beasts. Anyone could peddle any sort of food, drink, or medicine 
he could induce the public to buy. Anyone could do whatever he wished to 
the waters which adjoined or ran through his property, and to the air above 
it. 

Eventually, governments of industrial societies were obliged to move 
into the area of technical decision-making. Sometimes they were forced to 
do so by a catastrophe which made it dramatically self-evident that private 
interests could not be trusted to make technical decisions for the public 
welfare. More commonly, governments were moved, by degrees, after extended 
prodding by social reformers who contended that it was not enough to react, 


piecemeal, to isolated disasters: there were problems of broad population 
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groups being injured or made ill by hazards and contaminants on the job; 
poisoned by nostrums and unqualified healers; dragged down physiologically, 
psychologically, economically. The reformers called for institutional 
changes. 

Pure food and drug acts were passed; licensure laws covering physicians, 
dentists, hospitals, and other providers of health care; mine safety laws; 
restaurant sanitation ordinances; statutes of a hundred types undreamed-of 
in earlier times. But while passage of a local, state, or federal law might 
be the culmination of a political process, it was only the beginning of a 
technical process. To carry out a newly-enunciated public policy required 
a whole series of technical decisions. To say that foods, drugs, and cos- 
metics should be "pure" sounded straightforward enough; but someone had to 
define "purity" in myriad contexts, and continually redefine it as new pro- 
ducts appeared and more precise measurements developed. To say that physi- 
cians and dentists should be qualified was easy enough; but how was such a 
desideratum to be translated into concrete terms? To say that places of 
employment should be "safe" said almost nothing; what were the criteria to 
be? Where was the fine line to be drawn between an acceptable level of 
safety and an unacceptable level of hazard? 

Who was to make these decisions? To whom were they to be answerable? 
What was the process? What self-correcting mechanisms might it contain to 


take into account new evidence as it came to light? Many possibilities are 


imaginable; a number of them have been tried at various times, in various 


places, while some remain merely theoretical. 
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A. 


Decision-Making by Despots 


Some societies have attempted to engraft a superstructure of the new 
industrialization upon an old-style political substructure of authoritarianism. 
The ruler, or ruling clique, with power to pass judgment on economic, artistic, 
and all other matters, assumes ommicompetence to make scientific decisions 
as well. The results have usually been calamitous for the societies involved, 
and in some cases for millions of innocent bystanders. Hitler's ideas about 
biology were at the roct of the destruction of the Third Reich; agriculture 
in the Soviet Union has still not recovered from Stalin's notions about 
plant genetics. At best, totalitarian societies languish in the backwaters. 
For many years to come, Spain and Portugal will be paying for the scientific 
stultification that accompanied the political "stability" of Franco and 


Salazar. Despite the alleged "efficiency" of authoritarian systems, the 


record suggests that despotism and technological progress do not go well 


together. 
Decision-Making by Technician-Politicians 

Some forty years ago, a movement known as Technocracy, Inc., enjoyed 
a substantial following. Technocrats argued that the technologization of 
society had barely begun, that it would be a dubious boon to mankind so 
long as decisions were made by persons without technical competence, but 
an unmixed blessing if decision-making were by persons with special scien- 
tific and technical aptitude, training, and experience. The Technocratic 
plan was rather like a 20th Century version of Plato's Republic, with 
scarcely more likelihood of adoption. 

In an unorganized, attenuated form, however, the concept of the philo- 


sopher-king -- or, in contemporary terms, the technician-politician -- is 
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not beyond the realm uf the possible. Technically qualified persons could 
run for public office against lawyers, bankers, farmers, housewives, actors, 
clergymen, and others. Physicians and engineers are occasionally elected 
to the legislative branch, and when this happens they are almost always 
appointed to committees on which they can bring to bear what is assumed to 
be their special expertise. But physicians and engineers are not scientists. 
Scientists seem virtually never to run for office, outside of largely cere- 
monial positions in university communities. The reason may lie partly in 
the nature of the scientific temperament, which is uncongenial to the passion 
and hurly-burly of politics; and partly in the nature of popular attitudes, 
which incline to be skeptical of "eggheads'' who wander too far from their 
cloisters, or fearful of "Dr. Strangeloves'' thought to have uncanny powers 
and diabolical designs upon the universe. 
Decision-Making by the Demos 

The U.S. Constitution, as has often been pointed out, is republican 
rather than democratic in the strict sense of that word. It does not con- 
tain any provision for decision-making directly by the populace. Some state 
constitutions do, however, through the processes known as initiative and 
referendum. The same processes are available in many cities, counties, and 
special election districts. These processes may be, and with increasing 
frequency are, used for the resolution of technical issues. For example, 


in many local jurisdictions the question of whether to fluoridate the water 


supply, rather than being decided by the health department, city council, 


county supervisors, or directors of the water district, has been referred to 
the voters. Sometimes they approve, sometimes they disapprove, sometimes 


they change their minds. 


In California, in 1972, various environmental groups obtained enough 
qualifying signatures to place an omnibus Clean Air Initiative on the state- 
wide ballot. It was soundly defeated. In the same state, this year, an 
initiative dealing with nuclear safeguards was voted upon; it lost equally 
decisively. In Oregon, Washington, Wisconsin, and other states, decision- 
making by initiatives and referenda has sometimes had results more to the 
liking of the sponsoring environmental groups. 

The populist philosophy, which undergirds the initiative and referendum 
process, is attractive enough: since all the people are going to be affected 
in one way or another by broad environmental decisions -- e.g., whether to 
use public monies primarily for mass transit or freeways -- all the people 
ought to have the right to participate directly in such decisions; the 
initiative or referendum process affords the opportunity to educate the 
electorate in a way which the legislative process does not; a public policy 
will be more effective if it is well understood by the people whom it affects. 

In actual practice, environmental and other technical decision-making 
at the ballot box does not usually come close to the foregoing ideal. Measures 
tend to be drawn by ideologues rather than scientists; complexities tend to 
be reduced to the kind of simplistic sloganeering that can be summed up on a 
billboard or bumper sticker; and decision-making by direct ballot requires 
that issues be reduced to a dichotomy. Elections, by their nature, do not 


permit in-between positions, qualifications, negotiations. The voter must 


either accept a ballot proposition whole, or reject it entirely. Technical 


decisions are rarely so clear-cut. 


ys 
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Decision-Making by the Legislative Branch 


Strict constitutionalists might argue that the "republican" system 
created in 1789 is the best safeguard against demagoguery in environmental 


and other technical decision-making, and at the same time against usurpation 


of decision-making by any sort of elite. Carried to its ultimate conclusion, 


this point of view would call for legislative bodies not only to set broad 
public policy on environmental and other issues, but to fill in all the 
technical details and interstices. Presumably, legislators themselves 

would not do all the research required; the great bulk of it would be done 

by committee staffs in much the way the legislative process presently operates. 

Again, it is necessary to distinguish between what is theoretical and 
what is real. In theory, Congress could not only legislate the general 
principle that drugs should be unadulterated, effective, and safe when used 
as directed, but could promulgate lists of drugs which did and did not meet 
these tests. The role of the executive branch would then be limited to 
enforcement. This is perhaps not far from what the founding fathers had 
in mind; if citizens were aggrieved by technical decisions, they would have 
readier access to legislative decision-makers than to those in the labyrinths 
of the executive bureaucracy. 

But the reality is that legislative bodies are not equipped with the 
patiee lakintori ee, and other resources necessary for informed technical 
judgments, and the further reality seems to be that legislative bodies have 
no desire to take on such responsibilities. There is political peril in 
decision-making, and there appears to be special political peril in environ- 


mental decision-making. 


8 


In addition to everything else, a case can be made that technical 
decisions should be removed farther from, rather than moved closer to, the 
arena of political pressures. The founding fathers could not possibly have 
foreseen the complex scientific issues with which the Federal government 
would be contending in the 20th Century. If they had, they might well 
have concluded it was better for technical judgments to be made by civil 
servants than by congressmen more interested in the next election than in 
scientific truth. 

Decision-Making by the Judicial Branch 

Among the three branches of government, the one generally considered 
most dispassionate, most independent of political, economic, or other 
pressures, is the judiciary. Since dispassion and independence are among 
the hallmarks of science, it is sometimes thought that scientific and 
technical decisions can better be entrusted to the judiciary than to either 
the legislative or executive branches. It may be true that a court is 
usually more objective than a legislature, but that does not mean the 
judiciary is totally above the battle. As will be seen in a subsequent 
section, Federal agencies were propelled into pesticide safety regulations 
by court action brought by proponents of such regulations; when opponents 
did not like the results, they went to a court they suspected would be 
"friendly" and got them set aside. Which court acted out of scientific 


considerations? Probably neither. 


There seems an increasing tendency to go to court over environmental 


and other technical decisions, almost regardless of the issue, and the 
quality of the evidence on which it is based. Formerly, this was a prero- 
gative largely reserved for industrial associations, since court appeals 


are costly. But as environmental and citizens' groups gained in strength 
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and resources, they began making more frequent use of judicial processes: 
seeking writs of mandamus if they felt laws were being enforced improperly; 
seeking what they considered proper interpretation of the laws; appealing 
to higher courts if decisions by lower courts went against them. 

There can be no question these are protected rights of recourse avail- 
able to anyone who can hire attorneys or attract volunteer attorneys. But 
there can be a question whether scientific issues can best be settled in an 
adversary setting; a question whether the judicial system was intended to 
dispense technical wisdom as well as justice; a question whether the courts 
are any better equipped, or as well equipped, as the other two branches of 
government to do the amount and kinds of research requisite to well-considered 
technical decisions; a question, in short, whether the tendency to force the 
judiciary into technical decision-making is good for the clients affected by 
those decisions, good for science, good for the courts themselves, or good 
for society-at-large. 

Decision-Making by "Countervailing Forces"! 

Some might go so far as to hold that there is inevitably an adversary 
element in all environmental and other technical decision-making -- that some- 
body will be hurt, no matter how the decision goes -- and that therefore it 
is just as well to cut through all cant about dispassion and let the process 
be one of unvarnished struggle and power. Some labor unions, for example, 


do not care to rest occupational health and safety with government agencies; 


they feel they can do a more conscientious and effective job on behalf of 


their membership at the bargaining table. 
There are at least two shortcomings in this approach to technical 


decision-making. First, a decision on a matter such as occupational safety 
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ought not to be based on considerations so ephemeral as who has the power, 
at a given moment, to bring the other side to its knees. If five particles 
per cubic foot of air is a scientifically validated Threshold Limit Value 
for asbestos or some other contaminant, it adds nothing significant to 
worker health for a union to bludgeon management into a contract calling 
for one particle per cubic foot, and may merely force management out of 
business. On the other hand, if management happens to be in the ascendancy, 
it is obviously improper for a contract to permit twenty particles per cubic 
foot. 

Secondly, mechanisms simply do not exist for the resolution of most 
environmental and technical questions through the interplay of "countervailing 
forces". The process of collective bargaining between management and labor 
is extremely limited in its application, and has no counterpart elsewhere 
in society. Thus, for example, there is no way for nuclear power proponents 
and opponents to arrive at a decision between themselves. Their differences 
have to be referred to some other institutional arrangement for resolution; 
several possibilities have already been discussed above. Ina sense, 
Alternatives C, D, and E are variations on the ‘countervailing’ forces" 
approach. They may be thought of as forms of compulsory arbitration; after 


the contending sides make their presentation, a decision is rendered by the 


arbitrator -- either the public-at-large, the legislature, or a judge. 


G. Decision-Making by Administrative Agencies 


Leaving aside a variety of improbable or undesirable decision-making 


processes, and various possible combinations and permutations, there 
remains a major possibility: technical decisions may be arrived at through 
a lengthy process which begins with a legislative mandate, includes the 


opportunity for judicial review at the end, and, in between, involves 
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extensive staff and line work by an administrative agency -- frequently by more 
than one agency. 

This is a shorthand description of the way technical decisions are most 
often made in the U.S. at the present time. It is time consuming and cumber- 


some; it does not necessarily yield decisions which are scientifically 


defensible; sometimes it does not result in decisions which are defensible 


politically, economically, or in any other respect. But there is little 
point in speculating over radical changes at this time. For the foreseeable 
future, this is the principal method by which environmental and other technical 
decisions will be made if they are to be made at all. Since they must be 
made, and the need to make them will grow more overwhelming, not less, the 
most useful exercise for persons concerned about the quality of technical 
decision-making is to analyze the merits and demerits of present institutional 
arrangements, to build upon the strengths and attempt to correct the flaws. 
Decision-making by administrative agencies is consistent with the 
democratic ideal inasmuch as responsibility can ultimately be fixed, and 
if the citizenry is aggrieved it can call the responsbile official to 
account. True, a cabinet member or other appointed member of the administra- 
tion is not directly accountable to an electorate. But if his decisions create 
too great a storm of protest around the brow of the elected official who 
appointed him, he will probably not hold his position long enough to make 
many more decisions. And even if he rides out the storm, the administration 
of which he is a part will have to answer to the public at the next election. 
In some states and local jurisdictions, the citizenry need not even wait 
for the next regularly scheduled election. Jurisdictions which provide 
for the initiative and referendum usually add the process known as recall, 
whereby an elected official may be removed from office at any time a given 


proportion of the citizenry demands it. 
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All this is as it should be in a democratic society. But, as noted 
earlier, elected officials are almost never scientists, and, even if they 
were, they could not be expected to have competence in more than one field 
of specialization. In the very nature of the case, therefore, it is necessary 


for officials who bear the ultimate responsibility for technical decision- 


making to base their decisions on advice from others. Here, the technical 


and scientific staff of administrative agencies play the crucial role. They 
have the duty to provide the administrator with the information he needs to 
make the best decision possible under existing circumstances. If they 
provide information which is incomplete or, inaccurate, the administrator 
has the right and the duty to reorganize his agency. Let us turn now to a 
consideration of some of the sources of data, and data-evaluation, which are 
available to an administrative agency. 

II. Data-Sources in Technical Decision-Making 

A. Depending on its legislative mandate, an agency may require data 
from those who will be affected by its decision. It may require an Environ- 
mental Impact Statement, for example, from a company which proposes to build 
a new plant, or a builder who plans to build a new subdivision. An agency 
may require that a manufacturer of a new pharmaceutical or pesticide submit 
many kinds of data as a precondition of licensing the product for sale. 

Be To guard against the possibility that data furnished by a manu- 
facturer may be self-serving, agencies sometimes have the wherewithal to 
perform research of their own. They may use this capacity to check the 
replicability of work done by others, or they may perform original research, 


methodological or substantive. 
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C. An agency may review primary and ‘secondary sources which bear on 
the issue at hand: i.e., the background literature. Some large Federal 
agencies have brought this capability to a peak of high sophistication and 
refinement, making it possible to locate almost instantly all the references 
on a given subject in the scientific literature, not only in English but 
in most foreign languages. 

D. An agency may conduct a public hearing, or series of public hearings. 
The point of departure may be a proposed set of regulations, which give the 
appearance that a decision has already been reached. But the proposed regula- 
tions may be a mere formality; the real purpose of the hearings may be the 
development of informtion and interpretations which did not come to the 
attention of the agency in other ways. 

E. An agency may contract with an outside consulting firm or institution 
to examine some specific issue and gather certain relevant informaticn if it 
does not already exist; prepare a "state of the art" summary; draw up a set 
of guidelines or recommendations; or simply devise a data-gathering instrument, 
such as a questionnaire. 

F. An agency may appoint a "task force" or "working group" to address 
some ad hoc problem, and submit a report. Ideally, the members are chosen 
because of their special knowledge of the problem; in practice, professioral, 
regional, or other "balance" sometimes becomes a consideration. 

G. Several of the foregoing sources of data, and interpretations of 


« 


the data, involve what may be called "scientific input".> Scientists have 


Tat certain points in the discussion, for variety of expression, such 

terms as "scientific" and "technical", "science" and "technology", may 

be used almost as if they are interchangeable. They are not, of course. 
Science exposes; technology disposes. No environmental or other decisions 
of the type under discussion here are scientific decisions; nor are the 
decisions of peer groups. A scientist qua scientist might develop a new 
vaccine; a peer group might judge him worthy of financial support, and 
might in time recommend that the vaccine be used in a mass immunization 
campaign; an administrator might make the technical decisions as to who 
would give the injections, who would receive them, in what order of priority, 
on what date, and so forth. 
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the right of appearing at public hearings along with laymen; pre-registration 
data submitted by pharmaceutical manufacturers are presumably the work of 
scientists; consulting. firms may subcontract with chemists, physicists, 
biologists, sociologists, or other appropriate specialists; "task forces" 

are often comprised largely or wholly of scientists. But none of the 

sources mentioned to this point quite meets the criteria of the most 


disinterested, most systematic form of "scientific input" available. For 


want of a better term, that form will be referred to here as "peer review" 


III. The Meaning of Peer Review 


"Peer" in the sense of a nobleman is a usage peculiar to Britain, and is 
antithetical to our meaning here. "Peer", deriving from the Latin par, means 
one who has equal standing with another. When we say a person is judged by a 
jury of his peers, we mean that all persons are equal in the eyes of our juris- 
prudence. When we say that scientific data or conclusions are reviewed by 
peers, we mean that they are evaluated by persons at least as well qualified as 
the persons who generated the data or formed the conclusions in the first place. 

There may be a difference of opinion as to who is equally well qualified 
and who is less well qualified. Professional in-groups do not like to be 
judged by anyone who is not a member of the in-group; as Shaw put it, "Every 


' Thus, for example, physicians 


profession is a conspiracy against society.’ 
are extremely resistant to having their decisions reviewed by non-physicians. 

In matters limited strictly to patient care, this point of view is generally 
honored in Medicaid, Medicare, and all other health programs in our society. 

But to the extent that the physician functions in any other capacity, the 
conception of "peer" shifts. If he functions as a businessman, he may pro- 
perly be judged by accountants and others qualified to evaluate business affairs. 
If he undertakes medical research, his methods and findings may properly be 


assessed by physiologists, pharmacologists, statisticians, or others with 


expertise appropriate to his particular research problem. 


For our purposes, then, the word "peer" has mostly generic meaning; it 


is largely situational. The "peers" who are qualified to evaluate a decision 
in solid waste disposal are probably not the same as those best qualified to 
evaluate sewage disposal; peers qualified to judge a decision about chlorinated 
hydrocarbon pesticides are not necessarily qualified to judge decisions about 


organic phosphorus compounds. College and university degrees are no sure 
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indicator. A Ph.D. in a scientific field of inquiry is no guarantee that the 
holder is a scholar, has an open and inquiring mind, or even that he is 4 very 
good technician. Someone with an M.S, may be a better scientist, operationally; 
or, conceivably, someone with few formal scientific credentials may be the peer 
of a full professor. An attorney who specializes in environmental cases may, 
over the years, learn as much about the relevant disciplines as persons with 
graduate degrees in environmental science. 

For the moment, however, there exist no satisfactory means for appraising 
purely individual qualifications for peer status. If would be very difficult 
for an attorney, or novelist, or historian to enter the scientific pantheon. 
Normative rules of thumb are generally employed. A doctorate is considered a 
better qualification than a master's degree; a doctorate from Harvard or UC 
Berkeley is considered better than one from a smaller university; tenure on 
a faculty is usually more qualifying than experience in private industry; 
tenure on a prestigious campus is more qualifying than tenure on a less presti-~ 
gious one; publications in general are qualifying, but appearances in some 
journals is considered preferable to others. 

The second word in the phrase we are endeavoring to define is "review". 
The dictionary definition, again, is helpful, but not all-sufficient. For 
our purposes, the concept not only implies a critical study of work done by 


someone else, but that this critical evaluation be done on some kind of syste- 


matic footing. In a sense, a biochemist is conducting a form of "peer review" 


every time he takes off an hour or two from his other activities, thumbs through 
one of the journals in his field, and thinks to himself that a certain piece 

of research was well conceived and conducted, and another not so well done. 

Such opinions may be communicated haphazardly to colleagues over coffee, and 


eventually, at fifth or sixth remove, find their way to the persons who con- 
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ducted the research in question, or to those considering conducting similar 
research, and may be taken into account. But chances are the opinions will 
be lost and be of no effect. 

If that same biochemist were to write a letter to the editor of the 
journal stating his opinions and the reasoning on which they were based, it 
would come closer to "peer review". But there would still be many imponder- 
ables. The biochemist would be under no obligation to write regularly; the 
editor would be under no obligation to print the letters; the scientists who 
did the original research or were thinking of conducting similar research, 
might not read the letters. 

Peer review, as we shall employ the phrase, reduces imponderables to a 
minimum. The participants in the process do not evaluate the evidence or 
neglect to evaluate the evidence, as they please. If they agree to participate, 
they read what is necessary, attend whatever meetings are necessary, render 
their opinions in whatever form is necessary for the fulfillment of their 
commitment. To obviate the possibility of misconstructions, opinions are 
often put into writing; if made orally, they are usually recorded by a secre- 
tary, or at least paraphrased in the form of minutes, and the minutes circu- 
lated for correction of omissions or errors. 

Next, the process of orderly peer review requires that the opinions of 


the reviewing peer or group of peers be communicated without fail to some 


appropriate decision-making authority, and that the decision maker actually 


read or listen to these opinions. He may be an editor, an administrator, 

whatever. The Pesuge does not require that he accept the opinions. If it 
did, advisors would become decision-makers, and administrators would become 
mere figureheads. As noted before, there are sound reasons why this would 


not be feasible or desirable in our type of society. 
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But it is more than desirable in our type of society -- it is vital -- 
thst administrators who are responsible for making decisions about environ- 
mental and other technical issues not only receive the best available scienti- 
fic evidence and opinions, but that they understand it. Which suggests 
another element in the concept of peer review as we are here using the term; 
the process should yield a body of recommendations which are well organized, 
well documented. and couched in language comprehensible to the person or per- 
sons to whom addressed. 

Another element in peer review, -as comprehended here, is so important it 
cannot be overemphasized. The concept, like the concept of science itself, is 
inherently incompatible with secrecy and spurious "confidentiality". Discre- 
tion is sometimes appropriate to avoid needless personal embarrassment; it is 
legitimate, for example, that the scientist who submits a paper for publica- 
tion in a scholarly journal not be told the identity of his peer or peers who 
evaluate it, 

But there can be nothing secret in the contents of the paper; nothing 
withheld concerning the research on which it is based. A quintessential in- 
gredient in peer review, as we conceive it, is that it opens up the field 
of inquiry involved, and the methodology employed, to any and all other inter- 
ested scientists. Valid scientific evidence must be replicable, and there is 
no way for others to check the quality of the original research by repeating 
it save through full disclosure. 


For example, research concerning the toxicity in laboratory animals or 


human health effects _of some hazardous material should be considered unaccept- 


able for publication in a scientific journal, discussion at a scientific meeting, 
or any other form of peer review, if the name of the material or chemical com- 


position is suppressed or disguised as a "trade secret". Similarly, it should 
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go without saying that research is not scientific in any legitimate sense of 


the word, and should be rejected out of hand in any peer review process, if a 


portion of the data is withheld by the investigator because he deems it "irre- 
levant" or for whatever reason. It is for his peers to judge such matters. 
All the evidence must be in full view, and if some of it does not support the 
investigator's hypothesis, that is the way of science, and of peer review. 

The compelling necessity for openness in the peer review process -- the 
matchless virtue in the process itself -- is that it is self-correcting. To 
paraphrase Jefferson, error of method or interpretation may be tolerated where 
peers are free to combat it, through methods and interpretations of their own. 
There is a danger in the peer review process, if the peer group grows ingrown, 
rigid, snobbish, self-protective. The cure is to keep. the peer group itself 
open, even as the results of its deliberations should be open. 

There are other ways in which the peer review process may be deflected 
or subverted. Peers are not immune to cultural values, family pressures, or 
other extra-scientific influences upon their judgment. It is conceivable that 
@ group of peers -- an advisory committee to a Federal agency, let us say -- 
may happen to share a common distorting predisposition. In the short run, 
such @ peer group may influence the decision-making of the agency in the 
direction of one particular point of view or the other. But if their activi- 
ties are open to the scrutiny of colleagues outside that advisory committee, 
outside that agency, the process of peers reviewing peers may be expected to 
work in subtle ways to correct excesses of prejudice and partisanship. The 
lone scientist is a fiction; a scientist may almost be defined as one who 
continually sharpens and modifies his intellectual tools through contact with 
his fellows. 

At its best, then, peer review is self-correcting, and if employed appro- 
priately by administrators as part of the process of making environmental 


and other technical decisions, may be self-correcting for society as a whole. 


IV. Examples of Peer Review 


The following examples of the peer review process as it operates in various 
situations are intended to be illustrative rather than exhaustive. 

A. In the world of scholarly and technical journals, “peer review" has a 
specialized meaning. When an author submits an article, the editor or editorial 
committee decides on a referee or panel of referees who are felt to have 
special expertise in the area covered by the article. They are asked to judge 
whether the contribution should be published intact; published with minor modi- 
fications; published only after major rewriting; or rejected altogether. 
Customarily, the evaluation is not done "blind"; that is, the referees know 
the identity of the author. The author, however, is not told the identities 
of the referees. He knows only that they are supposed to be his peers. The 
system sometimes gives rise to suspicions of cronyism; sometimes a scientist 
working in a new field of inquiry, or in an unfamiliar way, may question whether 
referees from more conventional areas and methodologies are truly his peers. 
But, in general, the system works as well as any which can be envisioned. If 
it leans, it is more likely to lean in the direction of publishing some mar- 
ginal contributions rather than denying publication to those worthy of it. 

A variation on this form of peer review, and one which is even sounder in 
some respects, consists in the presentation of papers at scientific meetings. 

At major conferences and conventions, these are refereed in advance by a pro- 


gram committee. They are often, though not invariably, submitted subsequently 


for publication in a suitable journal. The advantage of presentations at 


scientific gatherings over all other forms of peer review is that the quality 
of the work is evaluated not only by an editor or small group of referees, 
but by an entire audience of the investigator's peers. At well-org nized 


scientific meetings, adequate time is allowed for discussion from the floor. 


There is often, in addition, a discussant or panel of discussants who have 
studied the paper, in advance, and are prepared to initiate the discussion 
and to stimulate it if it lags. 

B. The term “peer review" enjoys wide currency in the field of medical 
care, with another special, limited meaning. In this context, it refers to 
evaluation of the judgments of physicians, usually after the fact, by com- 
mittees of their fellow-physicians. Major hospitals, for example, have "tis- 
sue committees" which decide, on the basis of biopsy material, whether surgi- 
cal procedures were indicated. A ee whose judgment is repeatedly shown 
to be faulty will likely be spoken to privately; if his judgment proves faulty 


flagrantly and frequently, his hospital privileges may be withdrawn. Most 


hospitals also have committees which review admissions, length of stay, and 


discharges. 

Medical care programs financed in whole or in part by tax revenues are 
increasingly using peer review in an effort to ensure that quality of care is 
maintained, while, at the same time, preventing the waste of funds. 

C. The National Institutes of Health, and other Federal agencies in a 
position to disburse research grants, make extensive use of peer review both 
in the awarding of the original grants and in the supervision of research in 
progress. 

After administrative processing, an application is submitted to a "study 
section" ASE" of experts in heart disease, occupational safety and health, 
or whatever the field.of research in which the applicant proposes to work. 
Each application is first voted = yes or no,’and then assigned a priority 
rating, which determines the sequence for funding rather than a flat recom- 


mendation to fund or not to fund. The number of projects funded by the agency 


will depend on the money available; the agency is expected to begin with the 
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projects given the highest priority ratings, although under extraordinary 
circumstances, the administrator has latitude to overrule the judgment of 


@ study section. 


D. Elsewhere, peer review mechanisms may be less highly formalized, but 


there are no professions without some semblance of the process. A professional 
person's special competence is the product of professional peers. He is 
trained as a lawyer, or architect, or engineer, or political scientist, by 
lawyers, architects, engineers, or political scientists. Ina field requiring 
licensure, he is licensed by a board consisting largely if not entirely of 
professional peers. And it is hardly possible to have a career in any pro- 
fession without contacts with one's peers. In the course of these contacts, 

it is hardly possible to escape being judged. The judgments may be formal, as 
when an attorney violates the ethical codes of his profession, is tried by a 
committee of his bar association, and disciplined. At the very least, an infor- 
mal sort of peer review goes on constantly. A trial lawyer is judged highly in 
the eyes of his colleagues if he accepts difficult cases and wins them; he is 
judged poorly if he takes easy cases, and loses them. Even judges may be 
judged -- by the quality of their decisions, and by the frequency with which 


they are sustained by higher courts. 
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Vv. Forms of Peer Review Available to EPA 

The Environmental Protection Agency has a number of forms of peer review 
at its disposal to advise and assist in the decision-making process. Some 
have already been adumbrated in the more general discussion above. 

A. To the extent EPA dispenses research grants to independent investi- 
gators, it utilizes essentially the same system described in section IV-C. 

B. In the promulgation of all but emergency regulations, EPA conducts 
public hearings to which scientists are invited. While not a form of peer 
review, per se, any testimony which scientists give on such occasions invites 
scrutiny by their peers; their professional reputations to some extent are at 
stake. 

C. EPA's own scientific personnel -- for example, at Triangle Park -- 
conduct research, and write for publication. Certain in-house publications 
are not refereed by external peers. But anything submitted to a regular 
scholarly journal is, of course, subject to the peer review process described 
in section IV-A. EPA has ready access to all the relevant literature, by its 
own scientists, and others as well. Through a highly sophisticated reference 
retrieval systen. 

D. Although there is some jealousy and secrecy between agencies, and even 
within agencies, scientific work by EPA's staff is subject to review by peers 


in the Department of Labor, Department of Agriculture, Department of Interior, 


and other agencies with an interest in some specific environmental issue, just 


as the scientific work of these other agencies is presumably subject to review 
by peers in EPA. Coordination is available through such interagency groups as 
the Council on Environmental Quality. 

E. EPA is able, when it wishes, to convene open symposia, or invited- 


input symposia, at which qualified persons review the evidence bearing on some 
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decisions confronting the Agency, point out the lacunae in existing knowledge, 
recommend ways in which these lacunae might best be filled. 

F,. Under some of the laws which affect it, EPA is required to seek the 
advice of qualified advisory groups or of the National Academy of Sciences 
during the decision-making process. Under other laws, consultation with NAS 
is optional with the Administrator. For example, under a 1972 rewriting of 
the Federal Insecticide, Fungicide, and Rodenticide Act (sometimes known as 
the Federal Environmental Pesticide Control Act), "Upon the request of any 
party to a public hearing (following cancellation of a pesticide registration) 
or when in the Hearing Examiner's judgment it is necessary or desirable, the 
Hearing Examiner shall at the outset of the hearing refer to a Committee of 
the National Academy of Sciences the relevant questions of scientific fact 
involved in the public hearing. Each such committee shall include 
qualified scientists not more than one-third of which shall have an economic 
interest in the chemical industry. None of the members of such committees 
shall have any economic interest in che-oteristae which is the subject of 
referral. The Committee of the National Academy of Sciences shall report in 
writing to the Hearing Examiner within 60 days after such referral on these 
questions of scientific fact. The report shall be made public and shall be 
considered as part of the hearing record.” 

G. EPA makes extensive use of consultants and "task forces". To the 
extent they function sui generis, they are not a form of peer review (although 
their work may, in turn, be reviewed by other peers). But to the extent they 
openly appraise work already done, inside or outside EPA -- and that is often 
the principal thrust of "task forces" -- they may properly be regarded as a 
type of peer review. 

H. At its inception, on December 2, 1970, EPA took over Advisory Commit- 


tees on Environmental Radiation Exposure, Air Pollution Control Techniques, 
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Air Quality Criteria, and Air Pollution C hemistry and Physics from the Depart- 
ment of Health, Education and Welfare. On May 21, 1971, an Advisory Committee 
on Hazardous Materials was created by EPA Administrative Order No. 1130.3B. 
Five more Advisory Committees have subsequently been added: Ecology, Environ- 
mental Health, Environmental Measurements, Environmental Pollutant Movement and 
Transformation, and Technology Assessment and Pollution Control. 

Since January 11, 1974, all of EPA's Advisory Committees have been coor- 
dinated by a Science Advisory Board (SAB) with an executive committee consisting 
of the chairmen of the individual committees plus some members-at-large. SAB 
has a secretariat independent of the staffs of the individual committees. 

Among other things, SAB is intended to provide "a strong link between EPA's 
Administrator and the scientific community" and “independent review and advice 
on EPA's major scientific programs"... It is to "advise on broad scientific, 
technical and policy matters; make recommendations concerning needed research 


and development activities; assess the results of specific research efforts; 


assist in identifying emerging environmental problems; provide advice with 


respect to the Agency's relations with other governmental agencies, citizen 
groups, industrial groups, and educational institutions; and assure the cohesive- 
ness and currency of the Agency's scientific program." 

Taken together with the ten individual Advisory Committees, this seems 
tantamount to a charter for peer review of all EPA's scientific activities. 
Every one ‘of the essential ingredients appears to be present: members of these 
advisory groups are drawn from outside the Agency; they are authorized to advise 


« 


zone on Meteorology has been dropped; another, on Drinking Water Standards, 


has been replaced by a statutory committee under the Safe Drinking Water Act. 


SEPA Order 1130.30A. 
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on research which the Agency should support, in the manner of a study section; 
to evaluate the quality of work already in progress; to draw conclusions from 
the evidence, and make recommendations on policy matters, 
In 1975, SAB had a budget of $250,000; the Hazardous -Materials Advisory 
Committee a budget of $227,000 and six full-time staff positions; the Ecology 


Advisory Committee a budget of $110,000 and 3.5 staff positions; other Advisory 


Committees somewhat smaller resources. Each of the ten committees is chartered 


for fifteen advisors, but not all positions are filled at any one time. 

It would seem that the SAB is the capstone in a rich variety of peer review 
potentialities available to EPA for use in its scientific activities and techni- 
cal decision-making. Let us turn now to the question of how effectively EPA 


utilizes these potentialities. 


